\chapter{Installing HyperDex}
\label{chap:installation}

HyperDex provides multiple installation methods for a variety of environments.
Those looking to play around with HyperDex and go through the tutorials may find
it easiest to start with the quickstart Docker container.  For Linux users, the
easiest and most convenient way to install HyperDex is to use pre-compiled
binary packages.  OS X users have the option to install using Homebrew
packages.  Finally, HyperDex is also distributed as a set of tarballs so that
power users may install HyperDex in a custom manner on their platform of choice.

\section{Quick Installation with Docker}

HyperDex supports running within Docker containers for easy deployment.  With
Docker, an application is isolated in its own container, unable to change the
workstation outside the container.  This makes it easy to rapidly deploy new
applications without worrying about having to cleanup or revert any changes they
may make to the OS.  This sample environment is fully feature-complete, and
applications developed against such an environment will work with distributed
and sharded HyperDex clusters with zero application level changes.

For HyperDex, we can leverage containers to setup short-lived clusters that may
be used for one-off tasks, rapid prototyping, or for going through the HyperDex
tutorial---all without any commitment or modification to our base OS.  The
\code{hyperdex/quickstart} container includes everything necessary to run a
single-node HyperDex cluster, including the coordinator, daemon, and Python
bindings.

Running the HyperDex quickstart container is a simple three-step process:

\begin{enumerate}
\item We'll need Docker installed to use this container, which may be done by
following the \href{https://docs.docker.com/installation/}{Docker installation
instructions}.
\item We then fetch the HyperDex with the command \code{docker pull
hyperdex/quickstart}.
\item To start the cluster, run \code{docker run --net=host
-t -i hyperdex/quickstart}.  When all goes well, you'll see the following message
indicating your cluster is ready for use:

\begin{verbatim}
...
daemon: I1230 19:59:50.547586    21 daemon.cc:492] reconfiguration complete...
The transient HyperDex cluster is now online.

This is a transient HyperDex cluster.
You can connect to this cluster at address=172.17.3.27, port=1982.
\end{verbatim}
\end{enumerate}

With this quickstart cluster, both the coordinator and daemon logs will be
displayed on stdout.  You can connect to the cluster at the address and port
listed and can stop the cluster at any time using the command displayed on your
console.  When you stop the cluster, the HyperDex cluster will be destroyed, and
taken offline automatically.  To fully clean up any disk space in use by the
cluster, run \code{docker rm b6ae7c60c275}, where b6ae7c60c275 is the ID of your
quickstart container.

\section{Installing on CentOS/RedHat Using Binary Packages}

The following code snippet will install HyperDex on any 64-bit CentOS 6 or 7
system.  The script configures the HyperDex repository, imports the HyperDex
signing key, and installs the \code{hyperdex} package.  This script is available
in the HyperDex tarball or repository at \code{doc/install/centos-packages.sh}.

\inputminted[frame=lines,framesep=2mm,firstline=5]{bash}{\topdir/install/centos-packages.sh}

\noindent\textbf{Note:}  Users installing on Red Hat Enterprise Linux may have
to follow the instructions to install Extra Packages for Enterprise Linux
\footnote{\url{https://fedoraproject.org/wiki/EPEL}} prior to running the above
code.

\section{Installing on Debian Using Binary Packages}

The following code snippet will install HyperDex on any 64-bit Debian 7 (Wheezy)
system.  The script configures the HyperDex repository, imports the HyperDex
signing key, and installs the \code{HyperDex} package.  This script is available
in the HyperDex tarball or repository at \code{doc/install/debian7-packages.sh}.

\inputminted[frame=lines,framesep=2mm,firstline=5]{bash}{\topdir/install/debian7-packages.sh}

\section{Installing on Ubuntu Using Binary Packages}

The following code snippets will install HyperDex on 64-bit Ubuntu systems.
Pick the script that corresponds to the distribution you are running.  Each
script configures the HyperDex repository, imports the HyperDex signing key, and
installs the \code{HyperDex} package.  To figure out which version of Ubuntu you
are using, \code{cat /etc/os-release} and look for the corresponding version and
codename.

\subsection{Ubuntu 12.04 Precise Pangolin}

This script is available in the HyperDex tarball or repository at
\code{doc/install/ubuntu12.04-packages.sh}.

\inputminted[frame=lines,framesep=2mm,firstline=5]{bash}{\topdir/install/ubuntu12.04-packages.sh}

\subsection{Ubuntu 14.04 Trusty Tahr}

This script is available in the HyperDex tarball or repository at
\code{doc/install/ubuntu14.04-packages.sh}.

\inputminted[frame=lines,framesep=2mm,firstline=5]{bash}{\topdir/install/ubuntu14.04-packages.sh}

\section{Installing on Fedora Using Binary Packages}

The following code snippet will install HyperDex on any 64-bit Fedora 20 system.
The script configures the HyperDex repository, imports the HyperDex signing key,
and installs the \code{hyperdex} package.  This script is available in the
HyperDex tarball or repository at \code{doc/install/fedora-packages.sh}.

\inputminted[frame=lines,framesep=2mm,firstline=5]{bash}{\topdir/install/fedora-packages.sh}

\section{Installing on Linux Using Universal Binaries}
\label{sec:installation:universal-linux}

HyperDex is also available as a self-contained universal binary that works on
most glibc-based Linux distributions.  To use this installation, first download
the requisite tarball, and then you can install HyperDex like this:

\inputminted[frame=lines,framesep=2mm,firstline=3]{bash}{\topdir/install/linux-amd64.sh}

\section{Installing on OS X Using Homebrew}
\label{sec:installation:os-x-homebrew}

HyperDex is available on Mac OS X using the Homebrew package manager.  The
following script, available in the HyperDex tarball or repository at
\code{doc/install/homebrew.sh} may be used to install HyperDex on OS X:

\inputminted[frame=lines,framesep=2mm,firstline=3]{bash}{\topdir/install/homebrew.sh}

\section{Installing From Source}

Installing from source tarballs is a straightforward process that works on most
any recent Linux distribution.  Using the source tarballs provides power users
with control over the installation process and the opportunity to customize
the HyperDex installation.

To install from source, we must first prepare the system by installing a
compiler, and dependencies that are not managed by the HyperDex team.  This
process will be highly system specific, but should be familiar to most users who
install from source.  The HyperDex team and community are always happy to
provide assistance to users installing from source.

On systems like Ubuntu, we can prepare the system for the from-source install
with these commands:

\inputminted[frame=lines,framesep=2mm,firstline=3]{bash}{\topdir/install/ubuntu14.04-source-prereqs.sh}

With the prereqs installed, installing HyperDex from tarballs may be
accomplished with this script:

\inputminted[frame=lines,framesep=2mm,firstline=5]{bash}{\topdir/install/source-install.sh}

\subsubsection{Changing the Installation Directory}
\label{sec:installation:source:prefix}

By default, HyperDex installs files in \texttt{/usr/local}.  If you'd like to
install HyperDex elsewhere, you can specify the installation prefix at configure
time.  For example, to install HyperDex in \texttt{/opt/hyperdex}:

\begin{consolecode}
export PKG_CONFIG_PATH=/opt/hyperdex/lib/pkgconfig
./configure --prefix=/opt/hyperdex
\end{consolecode}

Check the \texttt{--help} option to configure for more ways to tune where
HyperDex places files.

%Dependencies for Python Bindings:
%
%\begin{itemize}
%\item \href{http://python.org/}{Python}: Version 2.6 or 2.7 with the
%    development headers installed.
%\end{itemize}
%
%Dependencies for Java Bindings:
%
%\begin{itemize}
%\item \href{http://openjdk.java.net/}{Java}: We test against OpenJDK 7.  Your
%    system must include \texttt{javac}, \texttt{jar}, and the JNI development
%    headers.
%\end{itemize}
%
%Dependencies for Yahoo! Cloud Serving Benchmark (YCSB):
%
%\begin{itemize}
%\item \href{https://github.com/brianfrankcooper/YCSB/wiki}{YCSB} The YCSB
%    distribution is a moving target.  We generally build against the latest Git
%    release.
%\end{itemize}
%
%\subsection{Configuring}
%\label{sec:installation:source:configure}
%
%HyperDex uses the Autotools to make configuration and installation as
%straightforward as possible.  After extracting the HyperDex tarball, you'll need
%to configure HyperDex.  The simplest configuration installs HyperDex in its
%default location (\texttt{/usr/local}) using the C++ compiler found on the
%system.  The configuration is performed in the directory extracted from the
%tarball and looks like:
%
%\begin{consolecode}
%./configure
%\end{consolecode}
%
%This basic configuration will configure the HyperDex daemon and native client
%library components to be built; however it omits several useful options for
%configuring HyperDex.  The rest of this section will highlight common ways to
%configure HyperDex.  Unless otherwise noted, all options should work well
%together.
%
%\subsubsection{Java Bindings}
%\label{sec:installation:source:java}
%
%HyperDex does not build Java bindings by default.  To enable the Java bindings,
%you must pass \texttt{--enable-java-bindings} to \texttt{./configure} like so:
%
%\begin{consolecode}
%./configure --enable-java-bindings
%\end{consolecode}
%
%If any of the prerequisites are missing \texttt{./configure} should fail.
%
%\begin{quote}
%Java bindings are currently unavailable in \HyperDexVersion.
%\end{quote}
%
%\subsubsection{Python Bindings}
%\label{sec:installation:source:python}
%
%HyperDex does not build Python bindings by default.  To enable the Python
%bindings, you must pass \texttt{--enable-python-bindings} to
%\texttt{./configure} like so:
%
%\begin{consolecode}
%./configure --enable-python-bindings
%\end{consolecode}
%
%If Python or its headers cannot be found, \texttt{./configure} will fail.
%
%\subsubsection{Yahoo! Cloud Serving Benchmark}
%\label{sec:installation:source:ycsb}
%
%HyperDex provides all the source code necessary to build a HyperDex driver
%for the YCSB benchmark.  If Java bindings are enabled, then YCSB can be built
%with \texttt{--enable-ycsb}.
%
%\begin{consolecode}
%./configure --enable-ycsb
%\end{consolecode}
%
%Note that YCSB must be in your Java \texttt{CLASSPATH}.  Configure cannot detect
%YCSB by itself.
%
%\section{Installing from Git}
%\label{sec:installation:git}
%
%Developers wishing to contribute to the development of HyperDex may build
%HyperDex directly from Git.  Building from Git is as straightforward as building
%from source tarballs, but requires a few extra dependencies and some setup
%before the \texttt{./configure} step.
%
%In order to build the HyperDex repository, you'll need to have the following
%utilities installed.  Most of these utilities are prepacked for Linux
%distributions.  Note that since these dependencies are only required for
%building from Git, they will not be detected at \texttt{./configure} time and
%instead \texttt{make} will fail with an error message.
%
%Required Dependencies:
%
%\begin{itemize}
%\item \href{http://www.gnu.org/software/autoconf/}{Autoconf} Used as part of
%    the build system.  Required for all builds.
%\item \href{http://www.gnu.org/software/automake/}{Automake} Used as part of
%    the build system.  Required for all builds.
%\item \href{http://www.gnu.org/software/libtool/}{Libtool} Used as part of the
%    build system.  Required for all builds.
%\item \href{http://www.gnu.org/software/autoconf-archive/}{Autoconf Archive}
%    Used as part of the build system.  Required for all builds.
%\item \href{http://www.freedesktop.org/wiki/Software/pkg-config/}{pkg-config}
%    Used as part of the build system.  Required for all builds.
%\item \href{http://flex.sourceforge.net/}{Flex} Used for building internal
%    parsers.  Required for all builds.
%\item \href{http://www.gnu.org/software/bison/}{Bison} Used for building
%    internal parsers.  Required for all builds.
%\item \href{http://cython.org/}{Cython} Used for building Python bindings.
%    Required for \texttt{--enable-python-bindings}.
%    Recommmended version: $\ge$ 0.15.
%\item \href{http://www.gnu.org/software/gperf/}{Gperf}  Generate perfect
%    hashes.  Used in the client library.
%\item \href{http://www.gnu.org/software/help2man/}{help2man}  Generate manual
%    pages with options.  Used for man-pages.
%\item \href{http://johnmacfarlane.net/pandoc/}{pandoc}  Convert manual pages
%    from markdown to man page format.
%\end{itemize}
%
%You'll need to build po6, e, BusyBee, HyperLevelDB Replicant, and HyperDex from
%Git, as the development version often introduces across repositories.
%
%For each of these repositories, you may build and install the code with:
%
%\begin{consolecode}
%autoreconf -i
%./configure
%make
%make install
%ldconfig
%\end{consolecode}

\subsection{Troubleshooting}
\label{sec:installation:troubleshooting}

This section walks through some common installation problems that might be
encountered while installing HyperDex from source.

\subsubsection{Missing \texttt{libtool}}
\label{sec:installation:troubleshooting:libtool}

If your system is missing \texttt{libtool}, you'll see an error message like the
following:

\begin{consolecode}
configure.ac:6: installing `./install-sh'
configure.ac:6: installing `./missing'
Makefile.am:48: Libtool library used but `LIBTOOL' is undefined
Makefile.am:48:   The usual way to define `LIBTOOL' is to add `LT_INIT'
Makefile.am:48:   to `configure.ac' and run `aclocal' and `autoconf' again.
Makefile.am:48:   If `LT_INIT' is in `configure.ac', make sure
Makefile.am:48:   its definition is in aclocal's search path.
Makefile.am: installing `./depcomp'
autoreconf: automake failed with exit status: 1
\end{consolecode}

To correct this issue, install the \code{libtool} package using your
distribution's package manager.

\subsubsection{Missing \texttt{pkgconfig}}
\label{sec:installation:troubleshooting:pkgconfig}

If your system is missing \texttt{pkgconfig}, you'll see an error message like
the following:

\begin{consolecode}
./configure: line 18348: syntax error near unexpected token `PO6,'
./configure: line 18348: `PKG_CHECK_MODULES(PO6, libpo6 >= 0.3.1)'
\end{consolecode}

To correct this issue, install the \code{pkg-config} package (sometimes called
\code{pkgconfig}) using your distribution's package manager.

\section{Verifying Installation}
\label{sec:installation:verify}

Once you have HyperDex installed, you should be able to view the built-in help.
If the following commands provide meaningful output, then it is very likely that
HyperDex is installed correctly and ready for use.

\begin{consolecode}
hyperdex help
hyperdex daemon --help
\end{consolecode}

\section{Upgrading to 1.6}
\label{sec:installation:upgrade1.6}

The 1.6 release introduces new features that are backwards incompatible with
previous releases, most notably a binary document format.  Before upgrading to
1.6, backup your data.  Once 1.6 is installed, copy your data into a
newly-deployed 1.6 cluster.

\section{Upgrading to 1.5}
\label{sec:installation:upgrade1.5}

HyperDex 1.5 is backwards compatible with the 1.4 on-disk format.  To upgrade
from 1.4 to 1.5, take a backup and restart the cluster from the backup.

\section{Upgrading to 1.4}
\label{sec:installation:upgrade1.4}

Upgrading to 1.4 is a two step process.  First, take a full backup of your
cluster using the provided backup tools.  Start the 1.4 cluster from the
backup.  This will upgrade the coordinator to 1.4, and the daemons will work
with the old data directory.

Alternatively take a backup, and restart the coordinator from the backed-up
coordinator state on the same address.  Then do a rolling restart of the
daemons.

The 1.4 release's on-disk format is backwards incompatible with the 1.3 and
prior releases.  HyperDex daemons will automatically upgrade to the 1.4 format
when first launched.

\section{Upgrading to 1.3}
\label{sec:installation:upgrade1.3}

Upgrading to 1.3 is a two step process.  First, take a full backup of your
cluster using the provided backup tools.  Start the 1.3 cluster from the
backup.  This will upgrade the coordinator to 1.3, and the daemons will work
with the old data directory.

Alternatively take a backup, and restart the coordinator from the backed-up
coordinator state on the same address.  Then do a rolling restart of the
daemons.

\section{Upgrading to 1.2}
\label{sec:installation:upgrade1.2}

The 1.2 release introduces new features that are backwards incompatible with
previous releases.  Before upgrading to 1.2, backup your data.  Once 1.2 is
installed, copy your data into a newly-deployed 1.2 cluster.
